TITLE OF THE INVENTION: 

SYSTEM AND METHOD FOR NODES COMMUNICATING IN A 
SHARED NETWORK SEGMENT 

CROSS-REFERENCE TO RELATED APPLICATIONS: 
[0001] This application claims priority of provisional patent application 
Serial No. 60/482762 entitled, "System and Method for Nodes, Preferably 
Routers, Communicating in a Shared Network Segment," filed June 27, 2003, 
the entire contents of which are incorporated herein by reference. 

BACKGROUND OF THE INVENTION: 
Field of the Invention: 

[0002] The invention relates to a mechanism, that is a system and method, 
for breaking live lock and supporting group keying and re-keying in routing 
system of networks, in particular IP based networks such as the Internet. 

Description of the Related Art: 

[0003] Routing protocols exchange network-prefixes (routes) during their 
routing update process. Commonly used deployed routing protocols are Open 
Shortest Path First (OSPF), Routing Information Protocol (RIP), Intermediate 
system and Intermediate Systems (IS-IS) for intra-domain routing and Border 
Gateway Protocol (BGP) for inter-domain routing. A recommended intra- 
domain routing protocol is OSPF; it uses both multicast and unicast channels 
for their update and flooding process. None of the routing protocols are 
designed with inbuilt security mechanism. For OSPF, security extensions were 
added later. These security extensions were not widely used due to lack of 
automatic mechanism and deployment issues. For example no automatic re- 
keying is provided for, and MD5 (Message Digest 5) which is now being used 
for authentication in OSPFv2 does not provide adequate security. Deployments 
are based on manual keying that leads to problems especially against replay 
attacks such as described in an article by Jerome Etienne, "Flaws in packet's 
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authentication of OSPFv2", IETF Internet-Draft, n http://off.net/~jme/ietf/draft" 
etienne-ospfv2-auth-flaws-00.htmr\ Group keying and multicast security are 
still open security problems. 

[0004] Routing protocols like OSPF use multicast and unicast protocols for 
exchanging LSA (Link State advertisements). OSPF supports three different 
mechanisms to use security extensions, namely no security at all, security- 
using password based scheme, or cryptographic checksum based on Message 
Digest 5 (MD5) for authentication. MD5 authentication provides higher 
security than plain text authentication. MD5 authentication uses a key ID that 
allows the router to reference multiple passwords, making password migration 
easier and more secure. 

[0005] OSPFv3 (OSPF version 3) does not have fields for security. When 
OSPFv3 is run on top of IPv6, all the security mechanisms to IPv6 are 
leveraged. Routing tables for OSPFv2 (version 2) and OSPFv3 are separate 
and they do not share anything in common. For IPv6 Authentication header is 
mandatory, IPv6 expects some mechanism to be present for establishing the 
Security Association (SA) and applying the SA credentials to the flows. It may 
use Internet Protocol security/Internet Key Exchange (IPsec/IKE) or any other 
security protocol to provide keys and SA information's. 

[0006] OSPF uses multicast on a shared segment to reduce the traffic 
overhead, and uses point-to-point communication for non-shared segment or 
network. For IPv6 authentication is mandatory, yet currently there is no 
defined scheme to perform secured group communication. Moreover in OSPF 
the problem expands in different directions on shared network as follows: 

[0007] First, OSPF routers are supposed to send a "hello" packet to their 
neighboring peers in multicast channel. The hello packet is used to inform the 
neighboring routers and to perform a discovery process, and needs also to be 
authenticated. This creates a problem. Before sending the packet, OSPF 
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routers need the keys and credentials to generate and authenticate the packets. 
[0008] The problem is that the OSPF routers should know in advance whom 
all the other OSPF routers in the same segment are and what keys to be used to 
authenticate their packets. This is a so-called live lock situation. A hello 
message is the only way for an OSPF node to advertise that it is an OSPF node 
and wants to participate in the OSPF protocol. Only legitimate routers are 
supposed to send hello packets and to participate in the OSPF protocol. 

[0009] A further problem is how should the OSPF routers create the SA's 
(Security Associations) for inbound and outbound traffic when they do not 
know their neighbors. 

[0010] OSPF routers, after receiving hello packet from the neighbors, will 
cooperatively perform designation router and backup designation routers 
selection process. All these messages have to be authenticated. This is 
currently performed by manual keying. The messages are pre-configured and 
the operations are made. OSPF supports very little resistance against replay 
protection and with manual keying there is no replay protection with 
IPv6/IPv4. OSPF is an interior routing protocol and is used internally within a 
single administrative domain. To detect internal attacks, Intrusion Detection 
Systems are deployed at hot-spots with high traffic. Since OSPF is with a 
domain, the only way to defeat against such attack is by placing Intrusion 
detection systems on all the segments but it may be costly for network operator 
to modify almost every routing element. 

[0011] A structure in accordance with the above description is shown in, and 
described with regard to Fig. 1. Fig. 1 illustrates OSPF message exchange in 
shared segment. Routers Rl to R6 are routers connected in a shared LAN. The 
routers may be a Designated Router (DR) and a Backup Designated Router 
(BDR). As shown in Fig. 1, router R6 sends a hello packet, or/and it may send 
router LSA update message in multicast channel. The designated router DR 
provides the summary LSA in another multicast address which all the OSPF 
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routers Rl to R6 in the LAN will receive. The backup designated router BDR 
becomes active when designated routers DR fail. OSPFv3 leverages all 
authentication mechanism to IPv6. Currently there is no automatic support for 
secured group communication and how to authenticate the OSPF packets. Due 
to these shortcomings, there is insufficient security. 

[0012] OSPFv2 and OSPFv3 are proposed to perform manual keying for 
authenticating routers/nodes. This has several severe limitations. For example, 
there is no provision of automatic re-keying, no replay protection, and there is 
no way to make sure that only OSPF legitimate nodes are exchanging the hello 
protocol message. 

SUMMARY OF THE INVENTION: 

[0013] The invention provides a method and communication system which 
includes a plurality of nodes, preferably routers, communicating in a shared 
network segment. At least one multicast channel in the shared network 
segment on which the nodes, preferably routers, can send multicast messages 
to the other nodes. A further specific multicast channel is provided on which 
the nodes can send specific start multicast messages to other nodes. A node 
which boots up or starts a protocol application, preferably a routing protocol 
application, sends a multicast start message on the specific multicast channel. 
Another node, preferably a router, receiving this start message validates the 
authenticity of the start message and may send a response message. 

[0014] According to another embodiment, a method performed in a 
communication system including a plurality of nodes communicating in a 
shared network segment and at least one multicast channel in the shared 
network segment, is provided. The method includes a step of sending 
multicast messages from nodes on at least one multicast channel to other 
nodes. The method includes a step of providing a further specific multicast 
channel for sending start messages by the nodes to the other node. The method 
also includes a step of sending a start message on the specific multicast 
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channel by using a start node, wherein the start node starts an operation or an 
application. The method further includes the steps of receiving at a receiving 
node the start message, and validating an authenticity of the start message 
upon receipt of the start message at the receiving node. 

[0015] According to a further embodiment, a communication system is 
provided. The communication system includes a plurality of nodes, and a 
shared network segment for communication between nodes of the plurality of 
nodes. The communication system also includes at least one multicast channel 
in the shared network segment on which the nodes can send multicast 
messages to other nodes, and a specific multicast channel on which the nodes 
can send start messages to the other nodes. The communication system further 
includes a start node for starting an operation or an application configured to 
send a start message on the specific multicast channel, and a receiving node 
for receiving the start message configured to validate an authenticity of the 
start message. 

[0016] According to another embodiment, a node for use in a system 
including at least one multicast channel on which the node can send multicast 
messages to other nodes is provided. The node is configured to send a start 
message on a specific multicast channel of a system when the node starts an 
operation or an application. 

[0017] Compared to the solutions that are based on the Public Key 
Infrastructure (PKI), one of the advantages provided the invention is that it 
does not involve any infrastructure. In addition, the public key infrastructure, 
PKI, method takes a longer time to implement until it becomes fully 
operational. 

[0018] Another advantage is that the invention does not need to change the 
OSPF protocol. It is not restricted to OSPF alone. It can be used where any 
group communication is required. It serves as a keying/rekeying framework to 
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the existing application. The invention provides a mechanism among the 
routing elements that needs only initial kick-start, afterwards the routing 
elements self-stabilize, authenticate and validate each other by themselves. 
Operator intervention is not necessary. The invention does not need manual 
keying and provides appropriate solution for mobile routers. 

BRIEF DESCRIPTION OF THE DRAWINGS: 

[0019] The objects and features of the invention will be more readily 
understood with reference to the following description and the attached 
drawings, wherein: 

[0020] Fig. 1 illustrates OSPF message exchange in shared segment 
comprising several routers; 

[0021] Fig. 2 shows an embodiment of the invention which includes a 
mechanism and functions for the jump-start, group keying and re-keying 
processes; 

[0022] Fig. 3 illustrates an example of a format depicting parts of a startup 
message, in particular of the header of a startup message, according to an 
embodiment of the invention; and 

[0023] Fig. 4 illustrates parts of a session key update message, in particular 
of the header of a session key update message, and of a session key update 
acknowledgment message according to an embodiment of the invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS: 
[0024] Embodiments of the invention provide a system and method allowing 
to break live lock and being able to support group keying and re-keying in 
routing systems of communication systems and networks, in particular IP 
based networks. 

[0025] Embodiments of the invention solve problems especially with regard 
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to security problems such as group keying and multicast security. Further, 
these or other embodiments of the invention address the group keying and re- 
keying issues related to routing protocol infrastructure namely OSPF. 

[0026] In more detail, the invention supports and provides a means and 
function to perform a jump-start for identifying the legitimate nodes that are 
willing to participate in group communication. The invention also supports and 
provides a mechanism and function to distribute the group-keys during a re- 
keying process. The invention further supports and provides a mechanism and 
function enabling nodes(s) to distribute and synchronize themselves during the 
re-keying process. 

[0027] The invention provides several advantages. For example, the 
proposed Jump-start mechanism enables the OSPF nodes to exchange secret 
keys out-of-band. By providing such mechanism only legitimate nodes will 
participate in the OSPF routing protocol and each OSPF routers a priori knows 
the existence of other routers in the domain. 

[0028] In the invention, designated router and back up designated router may 
become the caretaker for establishing and distributing the SA to other routers 
in the shared segment. 

[0029] The embodiments provide some advantageous features on the start-up 
sequence for the OSPF process to perform automatic group keying and re- 
keying. 

[0030] The invention not only applies to OSPF routers. The invention can 
also be applied to any form of group communication where there are N (N>=1) 
server and M clients communicating in a shared (and also non-broadcast) 
segment. 

[0031] Fig. 2 shows an embodiment of the invention which provides a jump- 
start mechanism and function, and a group keying and re-keying process. Fig. 



7 



2 illustrates the OSPF message exchange in the shared segment. Routers Rl to 
R6 are routers connected in a shared LAN, the DR is a designated router and 
the BDR is a backup designated router. 

[0032] First, the jump-start mechanism will be described. The jump-start 
means and function is based on the fact that the OSPF node listens on the third 
multicast "start-up" channel. The other two multicast channels are used by the 
OSPF for exchanging hello packet and LSA summary. If the node did not 
receive any message on the "start-up" channel, then it sends a "jump-start" 
message signed by its private key. By receiving these "jump-start" messages, 
the nodes can detect other available nodes in the segment. 

[0033] The invention, according to one embodiment which provides secure 
OSPF signaling, may be based on the use of IPsec SAs (Security Association) 
that are setup as a result of initial signaling between the nodes. The router may 
begin with a jump-start mechanism instead of sending out its Hello packets. 
This may be implemented by a change to the state machine of the routing 
protocol. Further details of the jump-start mechanism is described below. 

[0034] Before sending an initiating message, e.g. a hello packet, each OSPF 
router Rl to R6 has to know what keys and credentials have to be applied to 
the hello packet. For OSPF, hello packet is normally the first message 
generated by the OSPF nodes so there is no way to find out the other legitimate 
nodes in the current OSPF shared segment. This is called a live lock situation, 
where the OSPF routers want to send the hello packet, but to generate hello 
packet OSPF, the OSPF routers have to know what keys and credentials to be 
applied to that packet so that other nodes can authenticate the hello packet. 
Current IPsec or other security mechanism looks for Security Parameter Index 
(SPI) and destination address. If there is no Security Association (SA) defined 
for the inbound traffic, the packets are discarded. It is the responsibility of the 
sending node to generate the SA before sending the packet to the network. 
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[0035] To solve this live lock problem, in at least one or more or all of the 
described embodiments of the invention when the OSPF application process is 
started, the hello packets are not, or need not be exchanged as the first packets. 
The OSPF process listens by monitoring on a (e.g. third) multicast channel 
which is referred to here as start-up channel. The other (e.g. two) multicast 
channels are used by OSPF for exchanging hello packets and LSA summary 
information. 

[0036] Basically, the OSPF is an application process that runs on a router 
(node). For example when a router is powered on, the router bootups. Most of 
the time all the routing applications like OSPF, Routing Information Protocol 
(RIP) or Border Gateway Protocol (BGP) etc are started during the start-up 
time of the router. But it may be possible that the router can start the OSPF 
process at a later time. This will for instance be done when the administrator is 
performing some maintenance operation on the router. First the administrator 
might have stopped the running process and, after completing the maintenance 
procedure, the administrator can re-start the routing (OSPF) application by 
typing in a command at the router (node). In the described embodiments, it is 
hence the node or the process such as the OSPF process within the node which 
sends or receives the messages or listens to the channels etc. 

[0037] If the started OSPF process or node does not receive any message on 
the "Start-up" channel, then it sends a "jump-start" (Message 1) signed by the 
node's private key as shown in Figs. 2 and 3. The "jump-start" message or 
packet is a multicast message. Other nodes that are active, for instance have 
just started, will receive this message. 

[0038] Fig. 3 illustrates an embodiment of such a packet or message which 
may be called a jump-start message (or packet), live-lock breaking message (or 
packet), or OSPF "Start-up" channel message. This multicast message, and 
message exchange, helps the OSPF nodes in a shared segment to identify the 
neighboring peers. 
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[0039] For example, if routers Rl and R2 are assigned as the two OSPF 
nodes, the first router R2 starts and its routing application, e.g. OSPF 
application, sends a "jump-start" message in the start-up multicast channel 
(which represents an OSPF Startup Channel). Since there is no other OSPF 
node active, router R2 will keep on listening on the multicast channel and also 
sends the start-up packet at regular intervals. 

[0040] Assume at a later time that Router Rl is booted up (OSPF 
applications is started at boot-time automatically) and then it will send the 
OSPF jump start packet in the start-up channel. The key fields in the packet 
header are shown in Fig. 3 and described below. 

[0041] When the OSPF application is started in the Router 1 either at boot up 
or at a later time in the router, it sends a digitally signed packet that is the 
jump-start packet in the Multicast Start-up session. 

[0042] As shown in Fig. 3, the jump-start packet may include the fields: 

Source IP Address = Router- 1, 

Destination (Multicast) IP Address = "Start-up Channel Address", 
Random Value (or Time Stamp), and 
Digital Signature (DS1). 

[0043] The source IP address designates the sending node that is illustrated 
here in this example as router Rl. The destination address is a multicast 
address designating the multicast channel. 

[0044] The random value or time stamp may be used for verification of the 
digital signature DS. The time stamp, if included in the packet, can further be 
used by other nodes for checking whether a received jump-start packet is a 
recent packet and not an old packet to be disregarded. 

[0045] The Router Rl generates the digital signature (DS1), preferably as 
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described below: 

DS1 = Enc (Source Address, Random Value, Private Key of Router 1). 

[0046] Router R2 listens on the start-up channel, receives the jump-start 
packet, and verifies the digital signature DS1 thereof by forming a digital 
signature DS2 as follows: 

DS2 = Enc (Source Address, Random Value, Public Key of Router 1). 

[0047] Router R2 compares the DS1 and DS2 and decides whether Router 
Rl is valid and accepted by router R2 (and/or other routers receiving the jump- 
start packet and verifying it similar to router R2), when DS1 = DS2. 

[0048] When DS1 is not equal to DS2, Router R2 (and/or other routers 
receiving the jump-start message packet and verifying it similar to router 2) 
decides that Router Rl is not valid and does not accept router Rl. 

[0049] Note that during the installation of routers Rl and R2 valid 
certificates are installed in each of them. The above case describes a scenario 
where Rl is joining after R2 and R2 is validating the Rl certificate. Similarly 
Rl will also need to validate the R2 certificate (if needed). 

[0050] The above process is repeated when other routers R3 to R6 will start 
and join the multicast session. 

[0051] The Session key update process is described as follows. Assume that 
the designated router (DR) and backup designated router (BDR) are up and 
running and a new router such as R4 is booted up in the shared segment. The 
new router R4 will send a jump-start message or packet such as described 
above and shown in Fig. 3, and establish the SA with DR and BDR. 

[0052] After this process, the DR will generate a unique session key and 
distribute it in a session key update message shown in Fig. 4A to the BDR. The 
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BDR upon receipt of this message sends an acknowledgement message, such 
as shown in Fig. 4B, to the DR which informs the DR that the BDR has 
received and accepted the session key update message, and will use the 
updated session key. 

[0053] FIG. 4 includes Figs 4A and 4B and shows a session key update 
packet or message (Fig. 4A), and a session key update acknowledgement 
(ACK) packet or message (Fig. 4B). 

[0054] The session key update packet includes one or more, preferably all of 
the following fields: 

Source IP Address = DR, 
Destination IP Address = BDR, 
Message Type = Session key update. 
Time Stamp, and 
Session Key 

[0055] The session key update packet is encrypted. The session key update 
packet will be sent by the DR not only to the BDR but to all other nodes 
(joining in the multicast session), using their respective Destination IP 
Addresses in the Destination IP Address field of the session key update packet. 

[0056] The session key update ack packet is an acknowledgment, ACK, 
message, for acknowledging the session key change to the DR from the BDR 
and/or the individual routers. 

[0057] As shown in Fig. 4B, the session key update acknowledgment packet 
includes one or more, preferably all of the following fields: 

Source IP Address - BDR, 
Destination IP Address = DR, 
Message Type = Session key ACK, and 
Time Stamp. 
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[0058] When a router such as R4 acknowledges the session key change, the 
field Source IP Address of course indicates the IP address of this router. The 
ACK message of Fig. 4B generated and sent by the other routers is the same. 

[0059] One or preferably both of these new messages, "jump-start" packet 
and session key update packet, form part of the "Jump-start 11 mechanism 
according to an embodiment of the invention. This method is a new way to 
kick-off the peer routers. The method and system preferably include one or 
more of the following items: 

generation of "jump-start" packet or message, 
acknowledgment of such a packet, 

generation of a session key packet or message (or session key update 
message), and 

acknowledgement of this session key packet. 

[0060] Further, distribution of the session key by the DR and the BDR is also 
a feature of the embodiment of the invention. 

[0061] These messages may use underlying security protocol for e.g., 
validating the certificates, generating authentication packet etc. 

[0062] According to one embodiment the designated router DR is the only 
active node. In this embodiment, when the DR is the only router in that 
segment, the DR after sending the message 1 as shown in Fig. 2, if the DR 
does not receive any other "jump-start" message from other OSPF nodes Rl to 
R6, BDR, decides and declares that it is the only available node in the 
segment. The DR uses its public /private key pair and determines a random 
session key. This session key can be generated by several schemes. One 
possible way is to generate the key as follows: Key = Hash( Random Number, 
private Key, Public Key, TimeStamp). 

[0063] This key is used as credential and is applied on the hello packet either 
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for authentication or encryption (message 2). No other nodes will be able to 
read or decrypt the packet. Another possibility is to sign using the private key 
of the node and then generate the hello packet. Using the private key to 
generate hello packet may be time consuming since there is no other nodes. A 
better way may be to generate session keys. 

[0064] In one embodiment, the BDR or other nodes join the shared segment, 
that is these nodes are active nodes. Later when the BDR router is started, the 
OSPF application in the BDR performs the same procedure like the DR as 
described above. Namely, the BDR waits for a certain amount of time and 
then sends a "Jump-start" packet on the startup-channel. Since the DR is now 
still listening on the "start-up" channel, the DR validates the BDR's signed 
message (issued by a single administrative authority). Then the DR and the 
BDR engage in Internet Key Exchange (IKE) to generate two SA's (message 
3). One SA is for unicast communication that is conducted between the DR 
and the BR, and another SA is for multicast communications that can be used 
for hello packet and LSA updates etc. 

[0065] According to the group communication mechanism, if a node joins a 
group or a node leaves a group, the group-keys have to be changed and new 
keys have to be distributed. In this situation, the DR was using its self 
generated keys to send hello packet. Now the BDR has joined and it has 
generated a new session key for multicast channel. The BDR uses that key and 
generates the hello packet and LSA updates etc. 

[0066] Now if Router R4 starts, it sends a "jump-start" message, which the 
BDR and DR will receive. Both the BDR and DR are engaged in the Internet 
Key Exchange (IKE) with the R4 (messages 4 and 5) to generate a unicast SA 
for the DR-R4 and BR-R4 communication. Apart from this, the DR also 
generates a new session key (for multicast) during the same message exchange 
4; it then informs the BDR about the new group key using the unicast SA (DR- 
BDR), such as shown in Fig 4 A, 4B. This way any new nodes which come and 
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join the OSPF network will receive a new session key. It is the responsibility 
of the DR to generate a new group key and distribute to all other nodes. 

[0067] If more and more OSPF nodes are joining, then the DR generates a 
new group key and distributes it first to the BDR, then the BDR and DR can 
evenly distribute it to all other nodes. For example if one hundred OSPF nodes 
are present in a shared LAN, the BDR will distribute to the new group key to 
the first fifty nodes then the DR will distribute the new group key to the other 
fifty nodes. This reduces the distribution time and speeds up the operation. 

[0068] Overall the DR and BDR may be responsible for generating the SA 
messages for unicast communication. The DR is also responsible for 
generating the group-key and the DR has to distribute the group-key to the 
BDR first. Depending on the number of nodes, both the BDR and DR co- 
operatively distribute this group key to other OSPF routers using the respective 
unicast SA messages. The group-keys are refreshed periodically and the same 
keys can be used for both inbound and outbound because in the OSPF all the 
routers are clients and also server for generating the multicast message. 

[0069] According to the embodiments of the invention, the DR generates a 
session key also if it has decided that it is the only available node in the 
segment. Further, if a new node joins then keys are generated again. 

[0070] This feature contributes to increase the security against attacks. 
Basically, for example, most of the security devices have two keys, one key is 
a short term key and another key is a long term key. The long term key is kept 
within the router (pair of public key and private key and/or long term secret- 
key). 

[0071] Consider a case where only one secret key would be provided in the 
router, and that key is being used for secret communication. An attacker may 
sniff the traffic and store those packets. At a later time, the attacker can 
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perform some off-line analysis on those stored packet and may be able to 
retrieve the keys. If the attacker should succeed, that particular long term key 
and thus the whole system is now compromised. Typically the life time of the 
keys may vary from several months to several years in the Internet. So when an 
attacker extracts those keys, attacks are possible which may damage the 
system. 

[0072] To avoid this, long-term keys are used as secrets and stored only 
within the device. Using the long-term key, most of the short-term keys are 
derived. The life-time of the short term key is valid only for that session. 
Security protocol derives the short-term key from the long term key in an 
irreversible manner, meaning that there is no relation between the long term 
key and the short term key. 

[0073] Each time the short term key will be different and will confuse an 
attacker who is sniffing those packets. In addition to this the short term keys 
are also refreshed periodically at regular intervals. This is called as "re-keying 
mechanism". This completion protection scheme is called "Perfect Forward 
Secrecy". 

[0074] In the present embodiments, if the DR is the only router, it performs 
two tasks. First, it sends periodic Hello messages in the normal OSPF 
multicast channel and for this it uses a Short term key that is derived from the 
long term key (public/private key) pair. This is because if some other OSPF 
node is plugged in, it will see the hello packet with authentication header fields 
and it will not be able to decode and hence cannot participate in the secured- 
OSPF communication. 

[0075] Second, the DR periodically sends the "jump-start" message in the 
start-up channel digitally signed by its private key. 

[0076] The above features of the mechanism and functions are applicable to 
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most of the network elements in a routing system, preferably in the Internet, 
and are also applicable to the service and middle box equipments like routers, 
especially 3G-infrastructure network elements. 

[0077] Although preferred embodiments have been described above, the 
invention is not limited thereto and may also be implemented in other ways, 
e.g. by combining, in any arbitrary fashion, one or more features of one or 
more embodiments with one or more features of other embodiments. 
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